Constrained Optimal Control Problems Research Articles

A spectral iterative algorithm for solving constrained optimal control problems with nonquadratic functional

The numerical approximation of the solution to Hamilton-Jacobi-Bellman (HJB) partial differential equations (PDEs) in constrained optimal control problems (COCPs) with nonquadratic functional is studied in this investigation. Discretizing both of the space and time of HJB PDE by the Legendre collocation method, a nonlinear system of algebraic equations is obtained to find the expansion coefficients of the approximate solution. However, solving the system of nonlinear algebraic equations is generally relatively difficult and time-consuming. To improve the implementation of the method, we combine the Legendre collocation method with the policy iteration (PI) algorithm. This process leads to a sequence of linear systems of algebraic equations to find the expansion coefficients. The convergence results for the proposed method are provided. Moreover, it is shown that the PI algorithm for a class of constrained optimal control problems with odd nonquadratic functionals is mathematically equivalent to the quasi-Newton's iteration. Finally, the efficiency and accuracy of the method are demonstrated by solving a number of examples.

New Adaptive Mesh Refinement Strategy for Entry Guidance via Sequential Convex Programming

This study focuses on the entry guidance problem, which is a highly nonlinear and constrained optimal control problem. The entry guidance problem is formulated as a nonlinear trajectory optimization problem with six-degrees-of-freedom dynamics, path constraints, and boundary constraints. The formulated problem is discretized and solved using sequential convex programming (SCP) with successive linearization. However, during the discretization process, conventional collocation methods encounter a challenge with high-frequency jittering in the control and bank angle profiles near active path constraints. Because the considered problem especially involves tight path constraints, the solution jittering issue becomes severe. Therefore, this study proposes a novel mesh refinement strategy to address this problem. Unlike conventional mesh refinement strategies based on interpolation error, the proposed approach resolves the convergence and solution jittering issue of the SCP framework by linking the direct and indirect methods with the costate estimation under successive linearization. Along with the development, we introduce a novel costate estimation method and provide proof of its validity. We present numerical simulation results of the proposed method and conduct a comparison with conventional approaches. The results obtained demonstrate successful mitigation of the solution jittering issue and a significant improvement in solution accuracy regarding compliance with the necessary conditions of optimality.

Robust Constrained Multi-Objective Guidance of Supersonic Transport Landing Using Evolutionary Algorithm and Polynomial Chaos

Landing of supersonic transport (SST) suffers from a large uncertainty due to its highly sensitive aerodynamic properties in the subsonic domain, as well as the wind gusts around runways. At the vehicle design stage, a landing trajectory optimization under wind uncertainty in a multi-objective solution space is desired to explore the possible trade-off in its key flight performance metrics. The proposed algorithm solves this robust constrained multi-objective optimal control problem by integrating non-intrusive polynomial chaos expansion into a constrained evolutionary algorithm. The computationally tractable optimization is made possible through the conversion of a probabilistic problem into an equivalent deterministic representation while maintaining a form of the multi-objective problem. The generated guidance trajectories achieve a significant reduction of the uncertainty in their terminal states with a marginal modification in the control history of the deterministic solutions, validating the importance of the consideration of robustness in trajectory optimization.

Optimization over the Pareto front of nonconvex multi-objective optimal control problems

Simultaneous optimization of multiple objective functions results in a set of trade-off, or Pareto, solutions. Choosing a, in some sense, best solution in this set is in general a challenging task: In the case of three or more objectives the Pareto front is usually difficult to view, if not impossible, and even in the case of just two objectives constructing the whole Pareto front so as to visually inspect it might be very costly. Therefore, optimization over the Pareto (or efficient) set has been an active area of research. Although there is a wealth of literature involving finite dimensional optimization problems in this area, there is a lack of problem formulation and numerical methods for optimal control problems, except for the convex case. In this paper, we formulate the problem of optimizing over the Pareto front of nonconvex constrained and time-delayed optimal control problems as a bi-level optimization problem. Motivated by existing solution differentiability results, we propose an algorithm incorporating (i) the Chebyshev scalarization, (ii) a concept of the essential interval of weights, and (iii) the simple but effective bisection method, for optimal control problems with two objectives. We illustrate the working of the algorithm on two example problems involving an electric circuit and treatment of tuberculosis and discuss future lines of research for new computational methods.

Adaptive Dynamic Programming-Based Event-Triggered Robust Control for Multiplayer Nonzero-Sum Games With Unknown Dynamics1-4mmPlease verify and confirm the term "Multi-Player" has been changed to "Multiplayer" in the title of this article.

In this article, the event-triggered robust control of unknown multiplayer nonlinear systems with constrained inputs and uncertainties is investigated by using adaptive dynamic programming. To relax the requirement of system dynamics, a neural network-based identifier is constructed by using the system input-output data. Subsequently, by designing a nonquadratic value function, which contains the bounded functions, the system states, and the control inputs of all players, the event-triggered robust stabilization problem is converted into an event-triggered constrained optimal control problem. To obtain the approximate solution of the event-triggered Hamilton-Jacobi (HJ) equation, a critic network for each player is established with a novel weight updating law to relax the persistence of excitation condition based on the experience replay technique. Furthermore, according to the Lyapunov stability theorem, the present event-triggered robust optimal control ensures the multiplayer system to be uniformly ultimately bounded. Finally, two simulation examples are employed to show the effectiveness of the present method.

Improve generalization of driving policy at signalized intersections with adversarial learning

Intersections are quite challenging among various driving scenes wherein the interaction of signal lights and distinct traffic actors poses great difficulty to learn a wise and robust driving policy. Current research rarely considers the diversity of intersections and stochastic behaviors of traffic participants. For practical applications, the randomness usually leads to some devastating events, which should be the focus of autonomous driving. This paper introduces an adversarial learning paradigm to boost the intelligence and robustness of driving policy for signalized intersections with dense traffic flow. Firstly, we design a static path planner which is capable of generating trackable candidate paths for multiple intersections with diversified topology. Next, a constrained optimal control problem (COCP) is built based on these candidate paths wherein the bounded uncertainty of dynamic models is considered to capture the randomness of driving environment. We propose adversarial policy gradient (APG) to solve the COCP wherein the adversarial policy is introduced to provide disturbances by seeking the most severe uncertainty while the driving policy learns to handle this situation by competition. Finally, a comprehensive system is established to conduct training and testing wherein the perception module is introduced and the human experience is incorporated to solve the yellow light dilemma. Simulation results indicate that the trained policy can handle the signal lights flexibly meanwhile realizing smooth and efficient passing with a humanoid paradigm. Besides, APG enables a large-margin improvement of the resistance to the abnormal behaviors and thus ensures a high safety level for the autonomous vehicle.

Data-Driven Optimization for Deposition with Degradable Tools

We present a data-driven optimization approach for robotic controlled deposition with a degradable tool. Existing methods make the assumption that the tool tip is not changing or is replaced frequently. Errors can accumulate over time as the tool wears away and this leads to poor performance in the case where the tool degradation is unaccounted for during deposition. In the proposed approach, we utilize visual and force feedback to update the unknown model parameters of our tool-tip. Subsequently, we solve a constrained finite time optimal control problem for tracking a reference deposition profile, where our robot plans with the learned tool degradation dynamics. We focus on a robotic drawing problem as an illustrative example. Using real-world experiments, we show that the error in target vs actual deposition decreases when learned degradation models are used in the control design.

Optimal control approach for nonlinear chemical processes with uncertainty and application to a continuous stirred-tank reactor problem

Practical chemical process is usually a dynamic process including uncertainty. Stochastic constraints can be used to chemical process modeling, where constraints cannot be strictly satisfied or need not be fully satisfied. Thus, optimal control of nonlinear systems with stochastic constraints can be available to address practical nonlinear chemical process problems. This problem is hard to cope with due to the stochastic constraints. By introducing a novel smooth and differentiable approximation function, an approximation-based approach is proposed to address this issue, where the stochastic constraints are replaced by some deterministic ones. Following that, the stochastic constrained optimal control problem is converted into a deterministic parametric optimization problem. Convergence results show that the approximation function and the corresponding feasible set converge uniformly to that of the original problem. Then, the optimal solution of the deterministic parametric optimization problem is guaranteed to converge uniformly to the optimal solution of the original problem. Following that, a computation approach is proposed for solving the original problem. Numerical results, obtained from a nonlinear continuous stirred-tank reactor problem including stochastic constraints, show that the proposed approach is less conservative compared with the existing typical methods and can obtain a stable and robust performance when considering the small perturbations in initial system state.

A symplectic method for trajectory planning of general tractor-trailer systems

Trajectory planning is a key issue in the motion control of tractor-trailer system. In this paper, a unified differential-algebraic equation approach to establish the kinematic models of general tractor-trailer systems is developed. By analyzing practical physical limitations and transportation requirements, the trajectory planning of tractor-trailer system is formulated as a nonlinear constrained optimal control problem. Aiming at efficient numerical solution, a novel symplectic method is proposed. In the proposed method, a symplectic discretization is first implemented to transform the necessary conditions for the original optimal control problem into a set of non-smooth equations. Then, a composite strategy that integrates preconditioned iteration techniques is employed for robust solution. The proposed symplectic method is validated by three examples with different system configurations. Simulation results demonstrate that our method outperforms pseudospectral methods in computational efficiency and robustness to initial guesses. Specifically, a multi-stage strategy can be adopted to achieve robust solution in the presence of obstacles, making the system successfully travel across narrow corridors. An animation of the simulation results is available at www.bilibili.com/video/BV1EN4y1P7Ar.

Dimension reduction based adaptive dynamic programming for optimal control of discrete-time nonlinear control-affine systems

Dynamic programming based methods have been widely used in solving discrete-time nonlinear constrained optimal control problems. However, applying these methods in real-time is challenging because a large amount of memory is needed and the associated computational cost is high. Here, a search space dimension reduction strategy is proposed for a class of nonlinear discrete-time systems that are control-affine and invertible. Specifically, a bio-inspired motion rule is combined with inverse dynamics to reduce the value iteration search space to one dimension. The corresponding suboptimal control algorithm is developed and its optimality is analysed. An adaptation rule is developed to estimate uncertainties and improve the base policy. The closed-loop system is proven to be asymptotically stable. The advantages of the algorithm including much smaller computational cost and significantly reduced memory usage are demonstrated with two simulation examples.

Gradual-Impulsive Control for Continuous-Time Markov Decision Processes with Total Undiscounted Costs and Constraints: Linear Programming Approach via a Reduction Method

We consider the constrained optimal control problem for the gradual-impulsive CTMDP model with the performance criteria being the expected total undiscounted costs (from the running cost and the cost from each time an impulse being applied). The discounted model is covered as a special case. We justify fully a reduction method, and close an open issue in the previous literature. The reduction method induces an equivalent but simpler standard CTMDP model with gradual control only, based on which, we establish effectively, under rather natural conditions, a linear programming approach for solving the concerned constrained optimal control problem.

Application of the Alternating Direction Method of Multipliers to Control Constrained Parabolic Optimal Control Problems and Beyond

Application of the Alternating Direction Method of Multipliers to Control Constrained Parabolic Optimal Control Problems and Beyond

A Deep Reinforcement Learning Framework for Fast Charging of Li-Ion Batteries

One of the most crucial challenges faced by the Li-ion battery community concerns the search for the minimum time charging without damaging the cells. This goal can be achieved by solving a large-scale constrained optimal control problem, which relies on accurate electrochemical models. However, these models are limited by their high computational cost, as well as identifiability and observability issues. As an alternative, simple output-feedback algorithms can be employed, but their performance strictly depends on trial and error tuning. Moreover, particular techniques have to be adopted to handle safety constraints. With the aim of overcoming these limitations, we propose an optimal-charging procedure based on deep reinforcement learning. In particular, we focus on a policy gradient method to cope with continuous sets of states and actions. First, we assume full state measurements from the Doyle–Fuller–Newman (DFN) model, which is projected to a lower dimensional feature space via the principal component analysis. Subsequently, this assumption is removed, and only output measurements are considered as the agent observations. Finally, we show the adaptability of the proposed policy to changes in the environment’s parameters. The results are compared with other methodologies presented in the literature, such as the reference governor and the proportional–integral–derivative approach.

On necessary optimality conditions and exact penalization for a constrained fractional optimal control problem

AbstractThe purpose of this article is twofold. We first develop the first‐order necessary optimality conditions for a general constrained fractional optimal control problem using calculus of variation. These conditions are of the form of fractional ordinary differential equations which reduce to the conventional Euler–Lagrange equations when the dynamical system becomes a first‐order one. Then, we prove, via the optimality conditions established, that a popular penalty method used for conventional constrained optimal control and optimization is an “exact” penalty method for the fractional control problem by showing that an optimal solution to the penalized problem satisfies the optimality conditions of the original one when the penalty parameter is sufficiently large. An example is also provided to verify the necessary optimality conditions.

Cooperative signal-free intersection control using virtual platooning and traffic flow regulation

The emerging technologies of connectivity and automation enable the potential for signal-free intersection control. In this context, virtual platooning is posited to be an innovative, decentralized control strategy that maps two-dimensional vehicle movements onto a one-dimension virtual platoon to enable intersection operations. However, the effectiveness of virtual platooning-based control can be limited or degraded by parametric inaccuracies and unparameterized disturbances in vehicle dynamics, heavy traffic congestion, and/or uncoordinated platoons in multi-lane intersections. To explicitly address these limitations, this study proposes a hybrid cooperative intersection control framework consisting of microscopic-level virtual platooning control and macroscopic-level traffic flow regulation for traffic environments with connected autonomous vehicles. In virtual platooning control, vehicles approaching an intersection are organized into coordinated independent virtual platoons to avoid potential conflicts triggered by platoon formation changes. Through coordination, vehicles in a platoon are grouped into compatible passing sets to maintain desired safe spacing when proceeding through the intersection. We propose a distributed adaptive sliding mode controller (DASMC) which uses the backstepping control method and model reference adaptive control method to address parametric inaccuracies, and the sliding mode control method to consistently suppress the negative effects of the unparameterized disturbances. Each vehicle approaching the intersection utilizes the kinematic information from neighboring vehicles to implement the DASMC in a distributed manner such that vehicles within the same virtual platoon can achieve consensus safely. However, virtual platooning control cannot preclude excessive traffic from approaching the intersection, which can cause undesired spillbacks and degrade intersection control performance. To address this issue, traffic flow regulation is integrated with the virtual platooning control using an iterative feedback loop mechanism. In each iteration of the iterative feedback loop, a constrained finite-time optimal control (CFTOC) problem is solved to determine the optimal input flow permitted to proceed through the intersection, and the virtual platooning control provides feedback on the queue status to the CFTOC to initiate the next iteration. The effectiveness of the proposed intersection control framework is evaluated through numerical experiments. The results indicate that the proposed virtual platooning DASMC controller can mitigate the effects of parametric inaccuracies and unparameterized disturbances to achieve consensus for approaching vehicles, as well as guarantee string stability. Further, the proposed framework can alleviate traffic spillbacks and travel delays effectively through traffic flow regulation.

Model Predictive Control for Automotive Climate Control Systems via Value Function Approximation

Among the auxiliary loads in light-duty vehicles, the air conditioning system is the single largest energy consumer. For electrified vehicles, the impact of heating and cooling loads becomes even more significant, as they compete with the powertrain for battery energy use and can significantly reduce the range or performance. While considerable work has been made in the field of optimal energy management for electrified vehicles and optimization of vehicle velocity for eco-driving, few contributions have addressed the application of energy-optimal control for heating and cooling loads. This letter proposes an energy management strategy for the thermal management system of an electrified powertrain, based on Model Predictive Control. Starting from a nonlinear model of the vapor compression refrigeration system that captures the dynamics of the refrigerant in the heat exchangers and the power consumption of the system, a constrained multi-objective optimal control problem is formulated to reduce energy consumption while tracking a desired thermal set point. An efficient implementation of MPC is proposed for real-time applications by introducing a terminal cost obtained from the approximation of the global optimal solution.

Learning Stochastic Parametric Diferentiable Predictive Control Policies

The problem of synthesizing stochastic explicit model predictive control policies is known to be quickly intractable even for systems of modest complexity when using classical control-theoretic methods. To address this challenge, we present a scalable alternative called stochastic parametric differentiable predictive control (SP-DPC) for unsupervised learning of neural control policies governing stochastic linear systems subject to nonlinear chance constraints. SP-DPC is formulated as a deterministic approximation to the stochastic parametric constrained optimal control problem. This formulation allows us to directly compute the policy gradients via automatic differentiation of the problem's value function, evaluated over sampled parameters and uncertainties. In particular, the computed expectation of the SP-DPC problem's value function is back propagated through the closed-loop system rollouts parametrized by a known nominal system dynamics model and neural control policy which allows for direct model-based policy optimization. We demonstrate the computational efficiency and scalability of the proposed policy optimization algorithm in three numerical examples, including systems with a large number of states or subject to nonlinear constraints.

Optimal strategy for a dose-escalation vaccination against COVID-19 in refugee camps

<abstract> <p>An immunogenic and safe vaccine against COVID-19 for use in the healthy population will become available in the near future. In this paper, we aim to determine the optimal vaccine administration strategy in refugee camps considering maximum daily administration and limited total vaccine supply. For this purpose, extended SEAIRD compartmental models are established to describe the epidemic dynamics with both single-dose and double-dose vaccine administration. Taking the vaccination rates in different susceptible compartments as control variables, the optimal vaccine administration problems are then solved under the framework of nonlinear constrained optimal control problems. To the best of our knowledge, this is the first paper that addresses an optimal vaccine administration strategy considering practical constraints on limited medical care resources. Numerical simulations show that both the single-dose and double-dose strategies can successfully control COVID-19. By comparison, the double-dose vaccination strategy can achieve a better reduction in infection and death, while the single-dose vaccination strategy can postpone the infection peak more efficiently. Further studies of the influence of parameters indicate that increasing the number of medical care personnel and total vaccine supply can greatly contribute to the fight against COVID-19. The results of this study are instructive for potential forthcoming vaccine administration. Moreover, the work in this paper provides a general framework for developing epidemic control strategies in the presence of limited medical resources.</p> </abstract>

Mars Atmospheric Entry Guidance using MPSP with State and Control Constraints

An optimal Mars entry guidance scheme is presented in this paper using the Model Predictive Static Programming (MPSP) technique accounting for the applicable state and control constraints. The guidance scheme is designed to maximize the terminal parachute deployment altitude while applying minimum control effort and satisfying hard constraints on desired terminal conditions such as final velocity and downrange. The proposed guidance computes the optimal bank angle profile to shape the trajectory of the spacecraft. Path constraints on heat rate, dynamic pressure, aerodynamic load, and bounds on bank angle are considered to guide the vehicle safely through the martian atmosphere. Moreover, in order to generate practically realizable bank angle profiles, an additional constraint on the bank angle rate is also applied. Next, using the MPSP technique, the nonlinear constrained optimal control problem is converted into a static quadratic optimization problem with linear equality and inequality constraints to solve it in a computationally efficient manner. The concept of flexible final time MPSP is incorporated to update the final time in an optimal fashion. Numerical simulations illustrate the ability of the proposed method to solve the guidance problem efficiently while satisfying the path and terminal constraints within the desired accuracy.

Parametric variational procedure to solve index-3 semi explicit linear-regulator control problem with constrained robot illustration

The aim of this work is to solve 2-dimensional, 3-links linearized constrained robot problem. This problem is of index-three linear descriptor control problem. Since the aim is to regulate its output, a quadratic performance index is proposed to form an index-3 linear regulator optimal control problem. Firstly, the problem has been transformed into standard constrained optimal control problem over class of consistent initial condition with solvability assumptions. Secondly, the solution of differential-algebraic equation has been shown as equivalent to find the critical points of some variational formulation and vise- versa. Then aggregating all functional (performance one) with consistent initial conditions in one performance (weighted) index. Under standard topology of the solvability, a parametric approach has been proposed to approximate the state-space solutions with control over the class of constraints. Hence, an infinite dimensional problem is transformed into finite one, and then an approximate solutions is obtained by solving the class of linear algebraic equations. The proposed approach is easily implemented and accurate up to decision maker selections of the parameterization basis functions and their numbers.